In recent years, mobile computing devices have allowed users to apply image effects (e.g., image overlays, video effects) to one or more images captured via the client devices. Image effects can be applied to regions within a given image (e.g., recolor pixels of a person's face while leaving pixels of the person's hair unmodified). However, labeling different regions of a given image can be computationally intensive, especially on mobile devices which can be limited processing power and memory.